Bulk AG Overlap for bf16 on gfx950 - #713
Conversation
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
|
Reviewed the full diff of this PR against the base tip (16 files, +443/-63): the hipKittens bulk AG+GEMM kernel path ( Overall the approach looks sound — the Left 7 inline comments. The substantive ones are the new tests not actually distinguishing "fused kernel ran" from "fused kernel declined" (the reported name list reflects setup config, not the runtime gate), an unchecked shape coupling between the Userbuffers region and the dgrad GEMM in Copyright headers: OK. |
| # `fused_overlap_bulk_ag` is a CommOverlapP2P entry point | ||
| opts.p2p = True | ||
| elif opts.p2p: | ||
| warnings.warn("Point-2-point comms are not supported with bulk overlap.") |
There was a problem hiding this comment.
Consider updating the original warning message. Perhaps something like:
Non-fused point-2-point comms are not supported with bulk overlap
There was a problem hiding this comment.
The original warning message is from upstream, so I want to leave as is. I have added the missing IS_HIP_EXTENSION guards throughout the file to make it more clear this is a difference from upstream.
2 participants
Enables fused wgrad AG + dgrad GEMM for BF16 + gfx950.
Adds additional test coverage and layer tests, and makes fused/bulk_fused the default overlap scheme for gfx950, with fallback to no overlap at the moment for unaligned shapes, dtypes, and reduce-scatter.
1.282x Geomean speedup for wgrad AG + dgrad GEMM step